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1.  Introduction. 


One  of  the  important  problems  in  the  prediction  theory  of  multivariate 
stationary  stochastic  processes  is  to  obtain  some  algorithm  for  determinq  the 
best  linear  predictor  in  terms  of  the  past  observations.  Wiener  and  Masani 
[9],  [10]  solved  this  problem  for  the  full-rank  case,  when  the  spectral  density 
f  of  the  processes  is  bounded  above  and  away  from  zero,  in  the  sense  that 
there  exist  positive  numbers  c  and  d  such  that 

(1.1)  cl  <  f (e)  <  dl. 

Masani  L2]  improved  their  work  substantially  showinq  that  the  same  alaorithm 
is  valid  if  in  lieu  of  (1.1)  one  assumes  that 

(1.2)  (i)f^Loo  and  (ii)  f"1  e  L] . 

several  other  authors  proved  the  validity  of  the  same  aloorithm  under  more 
General  settings,  cf.  for  example  Salehi  [6J,  Pourahmadi  [8].  However,  all 
these  results  are  under  the  severe  restriction  of  full -rank  and  there  has  been 
no  extension  of  Wiener  and  Masani's  algorithm  beyond  the  full-rank  case. 

The  purpose  of  this  note  is  to  show  that  the  alaorithm  remains  valid 
exactly  in  the  same  manner  for  the  non-full -rank  processes  which  satisfy  the 
following  conditions 


(i) 

The  range  of  f(e)  is  constant  a.e.  (de), 

(1.3) 

(ii) 

f  €  Loo* 

(iii) 

f#  €  h  , 

where  A  stands  for  the  Generalized  inverse  (to  be  defined  later)  of  the  matrix 
A.  In  the  full-rank  case  these  conditions  clearly  reduce  to  the  conditions  (1.2).  and 


hence  our  result  generalizes  Masani's  algorithm  in  [2]. 

Masani's  assumption  and  approach  rests  on  a  characterization  (Theorem 
2.4,  [2])  for  full-rank  minimal  multivariate  stationary  stochastic  processes. 

Our  motivation  and  assumptions  are  based  on  a  characterization  of  jQ-regularity 
due  to  Makagon  and  Weron  [1].  We  will  employ  Wiener  and  Masani's  algorithm 
to  find  the  predictor  of  an  associated  full-rank  process  (to  be  clarified  later), 
which  is  produced  using  the  technique  of  Salehi  and  Miamee  [5],  and  usina  this  we 
will  obtain  our  algorithm  for  the  non-full -rank  process. 

In  section  2  we  set  down  the  necessary  preliminaries.  Section  3  is 
devoted  to  establishing  our  algorithm  for  determining  the  generating  function 
and  in  section  4  we  will  show  the  validity  of  Wiener  end  Masani's  alaorithm  for 
the  best  linear  predictor. 


2.  Preliminaries 


In  this  section  we  set  down  notations  and  preliminaries.  Most  of  these 
are  standard  and  can  be  found  in  [4],  [9]  and  [10].  Let  H  be  a  complex 
Hilbert  space  and  q  a  positive  integer.  H01  denotes  the  Cartesian  product  of 
q-copies  of  H,  endowed  with  a  Gramian  structure  as  follows:  For  any  two 
vectors  x  =  (x\...xq)^  and  y  =  (y\...,y^)^  in  their  Gramian  matrix  (x,y) 
is  defined  by 

(x,y)  =  [(xV)]^. 

It  is  easy  to  verify  that  it  has  the  following  properties: 

(x,.y)  >  0  ; 

(x,x)  =0  <~>  x  =  0  ; 
m  n  m  n 


where  X,Y,  X.,  Y,  are  in  Hq,  A.,  B-  are  constant  q*q  matrices,  and  A  >  0 

I  J  I  J 

means  A  is  a  non-negative  definite  matrix.  We  say  that  X  is  orthogonal  to  Y 
if  (X,Y)  =  0.  It  is  well  known  that  is  a  Hilbert  space  with  the  inner 
product 

( (X,Y) )  =  trace  (X,Y). 

A  closed  subset  M  of  Hq  is  called  a  subspace  if  AX  +  BY  e  M,  whenever  X  and  Y 
are  in  M,  A  and  B  are  q*q  constant  matrices.  It  is  easy  to  see  that  M  is  a 
subspace  if  and  only  if  M  =  M^  for  some  subspace  M  of  H.  For  any  X  in  Hq, 

( X | M)  denotes  the  projection  of  X  onto  M,  and  that  is  the  vector  whose  k-th 
coordinate  is  (X  |M),  which  is  the  usual  projection  of  X  onto  the  subspace  M. 


>v: 
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A  bisequence  Xp,  n  e  Z,  in  is  called  a  q-variate  stationary  stochastic 
process  if  the  Gramian  (Xm>  Xp)  depends  only  on  m  -  n. 

It  is  well  known  that  every  q-variate  stationary  stochastic  process  Xn 
has  a  non-negative  matrix  valued  measure  F  on  [0,2tt],  called  its  spectral 
measure  such  that 

(Xm  ’  V  =  4  f"  e’1(m  ’  n)edF^)* 

0 

f  stands  for  the  Radon-Nikodym  derivative  of  the  absolutely  continuous  (a.c.) 
part  of  F  with  respect  to  the  normalized  Lebesgue  measure  de,  and  it  is  called 
the  spectral  density  of  the  process. 

To  every  stationary  stochastic  process  Xn>  n  e  Z  the  following  subspaces 
are  attached: 

M(+«>)  *  sp  (X^,  -oo  <  n  <  »),  i.e.  the  subspace  of  H° 
generated  by  all  X^,  n  e  Z, 

M(n)  =  sp  (Xk,  -°°  <  k  <_  n), 

M(-oo)  =  nM(n), 
n 

M"(n)  =  sp  (Xk,  k  f  n). 

A  q-variate  stationary  stochastic  process  is  called 

(a)  non-deterministic  if  M(+°°)  f  M(n)  for  some  and  hence  all  n  in  Z, 

(b)  regular  if  M(-<»)  =  0 

(c)  minimal  if  M'(n)  l  M{+<»)  for  some  and  hence  all  n  e  Z, 

(d)  Jn-requ1ar  if  nM'{n)  =  0. 


If  X  is  non-deterministic  then  X„  l  M(n  -  1)  for  all  n,  and  hence  it  has 
n  n 

a  non -zero  one-sided  innovation  process 

<X„IH("  -  ’»• 

If  X  is  minimal  then  X  „  k  f-T(n)  for  all  n,  and  hence  it  has  a  non-zero 
n  n 

two-sided  innovation  process 

♦»  ’  Xn  -  <XnlM'("»- 

The  corresponding  one-sided  and  two-sided  predictor  error  matrices  are  defined 
by 

G  -  (gg»  gg)  and  z  =  (<b  g  »  ^  o  ^ 

respectively.  Xv  =  ( X^ | M(0) )  is  called  the  best  linear  predictor  of  log  v. 

Clearly  Xn  is  non-deterministic  if  and  only  if  G  f  0  and  minimal  if  and  only 
if  T.  f  0.  A  non-deterministic  (regular)  process  Xn  is  said  to  be  non-deterministic 
(regular)  of  full -rank  if  G  is  invertible.  The  process  is  called  full -rank 
minimal  if  it  is  minimal  and  its  two-sided  predictor  error  matrix  £  is 
invertible. 

It  is  useful  to  note  that  we  have  the  following  inclusions  between  these 
various  classes  of  processes 

non-deterministic  ±  regular  ±  minimal  ±  Jg-regular  f  full-rank  minimal. 

The  last  inclusion  is  a  consequence  of  Theorems  1  and  2  below,  and  the 


others  can  be  easily  verified. 
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It  is  known  that 

00 

M(n)  =  I  sp  (g  .  )  +  M(-®) . 
k=0  n_K 

Consider  G  as  a  linear  operator  on  to  C^,  C  being  the  complex  plane. 

Let  J  be  the  matrix  of  the  projection  on  onto  the  range  of  G,  and  we  nut 
(/£  +  J1)"1  =  H.  The  normalized  one-sided  innovations  are  defined  by  hp  =  Hgn- 
One  can  show  that  [4] 

X"  -AS  +  (xnlM( -'»’)• 

although  A^'s  in  this  decomposition  are  not  unique,  the  coefficients  are 

in  fact  unique  and  this  enables  us  to  associate  the  following  function  to  our 
process 

Ak^eike. 
k=0  * 

this  is  called  the  generating  function  of  the  process. 

We  shall  be  concerned  with  the  class  Ip  (1  <_  p  <_<*>)  of  all  q*q  matrix 
valued  functions  a  on  [0,2tt]  whose  entries  are  in  the  usual  Lebesgue  space 
Lp.  L°+  will  denote  the  subspace  of  L2  consisting  of  those  matrix  valued 
functions  whose  n-th  Fourier  coefficient  vanishes  for  n  <  0,  i.e. 

e"in6g(0)de  =  0,  for  all  n  <  0. 

U 

For  any  q*q  matrix  A  there  exists  a  unique  q*q  matrix  A  such  that  [7] 

AA*A  =  A,  A#AA#  =  A# 

(A#A)*  =  (A#A),  (AA#)*  =  AA*. 


This  matrix  A*  is  called  the  generalized  inverse  of  A  and  has  the  following 
further  properties 

NX(A)  =  R(A#),  R1(A)  =  N(A#), 

where  R(B)  and  N(B)  denote  the  range  and  null  space  of  the  matrix  B,  respectively. 

For  the  ease  of  reference  we  state  the  following  two  theorems  which  are 
due  to  Masani  l_2],  and  to  Makagon  and  Weron  [1],  respectively. 

Theorem  1 .  Let  Xn,  n  €  Z,  be  a  q-variate  stationary  stochastic  process 
with  spectral  distribution  F.  Xp  is  full-rank  minimal  if  and  only  if  F 

is  a.c.  and  its  spectral  density  f  is  invertible  with  f"1  e  L-j . 

Theorem  2.  Let  Xn,  n  e  Z  be  a  q-variate  stationary  stochastic  process 
with  spectral  measure  F.  The  process  Xn  is  JQ-regular  if  and  only  if 

(i)  F  is  a.c.  with  respect  to  de,  with  spectral  density  f, 

(ii)  R(f(9))  is  constant  a.e.  (de), 

,<44,  J  , 


3.  Determination  of  the  generating  function 


In  this  section  we  give  an  algorithm  for  determining  the  Generating 
function  of  a  (not  necessarily  full-rank)  stationary  stochastic  process.  The 
result  of  this  section  extends  Masani's  algorithm  developed  in  1.2]  to  the 
non-full-rank  case.  Our  technioue  is  essentially  that  used  by  Salehi  and 
Miamee  in  [5]  where  the  following  formula  for  the  two-sided  prediction  error 
matrix  I  of  a  Jg-regular  process  was  obtained 

2  -  [^r  [  ^(9)d9]#. 

0 

We  will  continue  this  work  under  the  assumption  that  our  process  is  JQ-reqular 
or  eguivalently  assuming  that  conditions  (i),  (ii),  and  (iii)  of  Theorem  2  are 
valid.  Let  h-j,  h2,  . ..,hp,  hp+i»--**hq  be  an  orthonormal  basis  for  the 
g-dimensional  complex  Euclidean  space  Cq  such  that 


R  =  R(f (0) )  =  sp  (h..,  1  <  i  <  p)  a.e.  (de). 


N  =  Rx  =  N( f (6) )  =  sp  (h.j ,  p+1  £  i  £  q). 

Let  e^ ,  e2,...,  e^  be  the  standard  basis  of  Cq.  Define  the  unitary  operator 

U  on  Cq  by  Uh^  =  e^,  1  <  i  <  q.  Letting  R-j  =  sp  (e^ ,  1  <  i  <  p)  then 

r|  =  sp(e.j,  p+1  £  i  £  q).  Clearly  U  maps  R  onto  R-|  and  R1  onto  R|  and  U*  maps 

R1  onto  R  and  R|  onto  r!  As  usual  we  will  identify  any  linear  operator  on  C° 

with  its  matrix  with  respect  to  the  standard  basis  of  Cq.  By  our  choice  of 


U  we  have 


(3.1) 


Uf(e)U*  =  0 

.0  0. 


^  "u  \  ^  .  •-  -w  .  "w  i  •  «r 
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where  g(Q)  is  a  p*p  non-negative  matrix  valued  function  whose  rank  is  a.e. 
equal  to  p.  Let 

Vn  =  UXn,  n  <r  Z 

be  a  new  stationary  stochastic  process,  then  we  have 

2tt 

(Ym’Yn}  =  (UXm’UXn)  =  U(27  j  ^e'1' (nwi)6f (e)de)U* 

0 


(3.2) 


1 

2tt 


[  e"^m_n^eUf(6)U*de 


0 


-  _L  2  p-i(m-n)0 

2tt  j 

0 


0 

0 


de. 


This  shows  that,  for  p+1  £  k  £  q,  the  k-th  component  of  Yp  is  zero  for  all 

n  p  Z.  The  p-variate  statioanry  stochastic  process  Zn  =  (Y^,. . . ,Y^)T  has 

spectral  density  g.  Since  U  takes  R  onto  Rj  and  Rx  onto  r|,  one  can  see  that 

# 

(3.3)  r>  u|  =  |y  u  =  (UfU*)#  =  U*f#U. 


q"1  01 

II 

g  0 

0  0 

. 

0  0 

u 

Mow  since  Xn  is  assumed  to  be  Jg-reqular,  Theorem  2  implies  that  f  (e)  is 
integrable.  Thus  (3.2)  implies  that  is  intearable  and  hence  by  Theorem  1, 
Zn  is  full -rank  minimal. 

We  are  going  to  utilize  Masani's  algorithm  to  obtain  the  generating 
function  T  and  predictor  of  this  full  rank  minimal  process  Zn,  and  then  use 
this  to  net  the  generating  function  $  and  predictor  of  our  process  X  .  The 
following  lemma,  which  reveals  the  close  tie  between  y  and  <j>,  is  crucial  in  the 
development  of  our  algorithm. 


Lemma .  Let  Xn,  n  e  Z  be  a  Jg-reqular  stationary  stochastic  process  with 
spectral  density  f.  Let  g  be  the  spectral  density  of  the  correspondina 


full -rank  minimal  process  Zp  discussed  above.  If  4>  and  h*  are  the  aenerating 


functions  of  X„  and  Z  respectively  then 
n  n 


$  =  U* 


where  U  is  the  unitary  matrix  obtained  above. 

Proof.  We  first  note  that,  since  $  and  ¥  as  generating  functions  are 


(3.4) 


f  =  U* 


- 

9 

0 

T  0 

Y  0 

U  =  (U* 

U)  (U* 

0 

0 

- 

0  0 

• 

0  0 

on  the  other  hand 


f  =  4>$*. 


Since  f  has  two  factors  0  and 

5  =  U* 


belonging  to  L^j  ,  to  complete  the  proof  it  suffices  to  show  that  the  latter  one 


is  also  optimal  (cf.  uniqueness  Theorem  4.4  of  [3]).  To  prove  this  we  first 
note  that  since  the  0-th  coefficient  V+(0)  of  T  is  nonnegative  definite  and 

. -  kt°)  °1 


5+(0)  =  U* 


we  have 


V  V  >  V  *.*  •  ■  •>  V  *>  "> 


\v.*. \v v-  .*•  .vv- 


On  the  other  hand  if 


(3.6) 


f  =  YY* »  Y  e  L 


0+ 


is  another  factorization  of  f  ,  then 

g  0 
0  0 

but  o  =  ¥¥*  implies  that 


(3.7) 


=  UfU*  =  (UyU*)(UyU*)* 


(3.8) 


g  0 

II 

'»  o' 

-6 

o  , 

o' 

o 

■ 

0  0 

p  oj 

Since  ¥  is  the  generating  function  of  Zn  one  can  prove  that  the  function 


¥  0 

0  0 


is  the  generating  function  of  Yp.  In  fact  we  know  that  the  generating 
function  $  of  a  q-variate  stationary  stochastic  process  Xn  is  given  by 

$  =  l  A  /I  ein6, 
n=0  n 

where  An’s  are  the  coefficients  in  the  representation 

X0  ■  LAns-n  + 


of  Xn  in  terms  of  its  innovation  process 

g„  =  xn  -  (xn|M(n-l)> 

and  G  =  (oq,qq)  is  the  predictor  error  matrix.  Comparing  7-n  with  =  [Z 
we  note  that 


zo  \l0K%  +  <Z0IH 


oj  L 


*2  °  *2 


n=0  0  0  j  [  0 


+  (Y0(MY(-oo)) 


Although  the  coefficients  arising  in  this  sum  are  not  unique  they  will  give 
us  the  generating  function  uniquely,  and  we  have 


.  fk  ^  i 

i  "  jF 
n~°  [  0  0 


.  r*2  «|  r^r  ol  .  „  - 

■  i  e_ine  ■  i 

n=0  0  0  0  0  ^ 


An  ^  0  e-ine 


l  AZv4^  e1n6  0 
n=0 


ine  0  $Z  0 


"  w  L  o  o 

$Z  0  to 

0  0  0  0 


Thus  is  the  optimal  factor  of 

0  0 


optimality  of 
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(3.7)  and  (3.8)  together  with  the 


*  •  •  •  >  v  *>•***• 

v'*.  'AO  %*  O 
•  * 


imply  that 


'F.(O)  o  fV(0)  o' 

+  >  (Uy.(0)U*)(Uy+(0)U*)*. 

LO  o  o  o  +  + 


This  in  turn  implies  that 


(6+(0))2  =  (U*  H'+(0)  0  U)  (U*  ’P+(0)  0  U)  >  Y+(0)Y+(0)*. 

[_o  oj  |_°  °J 

This  together  with  (3.5)  shows  that  6  is  the  optimal  factor  of  f.  Thus  by 


the  uniqueness  theorem  mentioned  above 


4>  =  6  =  U* 


\  O' 

0  0 


Q.E.D. 


Now  we  are  ready  to  give  the  algorithm  determining  the  generating  function 
of  our  JQ-regular  q-variate  stationary  stochastic  process  Xn>  Since  f  satisfies 
the  conditions  (i),  (ii),  and  (iii)  of  (1.3)  one  can  see  that  these  imply  that 
g  satisfies  the  corresponding  conditions  (i)  and  (ii)  of  (1.2). 

Thus  we  can  use  Masani’s  algorithm  developed  in  section  4  in  [2] 

to  compute  the  generating  function  $  of  the  desired  process  Xn  via  the  formula 

V  Ol 

$  =  U*  U 

0  0 


Remark.  One  can  similarly  extend  the  other  available  algorithms  (such 
as  that  in  [8])  to  obtain  corresponding  algorithms  for  the  non-full-rank  case. 


4.  Determination  of  the  Predictor 


In  this  section  we  show  that  the  unique  autoregressive  series,  of  [2], 
giving  the  linear  predictor  in  the  full-rank  case,  can  be  used  to  obtain  the 
predictor  in  our  non-full -rank  case.  In  fact  as  we  will  see,  exactly  the 
same  formula  works  in  this  case  as  well.  We  continue  to  assume  that  the 
density  f  of  our  stationary  stochastic  process  Xn  satisfies  conditions  (1.3). 
Using  the  notations  and  results  of  section  3  we  know  that 

U, 


and  the  density  g  satisfies  conditions  (i)  and  (ii)  of  (1.2).  Thus,  using  the 
technique  developed  in  [2]  one  can  show  that 


kz-k*  inH‘ 


k=0 


where 


Evk  =JQ  Cv+nDk-n 


.-1 


with  Ck  and  Dk  being  the  k-th  Fourier  coefficients  of  V  and  V  respectively. 
Now  one  can  easily  verify  that 


’Jo 


Evk  0 
0  0 


Y_k  ,  in  Hq, 


and 

(4.1) 
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_ 1 

n*0 

L  o 
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Since  Y  =  UX„,  one  can  also  verify  that 


Hence  we  have 


(4.2) 


E  .  0 

X_  =  U*  (  l  vk  Y  ) 
n  k=0  0  0  "K 


°°  k  ^ 
l  (U*  U)U*Y 

k=0  L  0  0_ 


in  Hq. 


Letting  Fyk  to  be 
(4.3) 


PU  ol 


F  .  =  U* 

vk  L  o  o_ 


we  get  the  following  autoregressive  series  representation  for  the  best  linear 


predictor  X^: 


xv  \L  v-t 


V  ki0  vk  -k  • 

Now  let  us  examine  the  coefficients  F^  in  (4.3)  more  carefully.  Doinq  this  we 
will  be  able  to  write  Fyk  in  terms  of  the  Fourier  coefficients  of  the  generating 
function  $  of  our  original  process  Xn  rather  than  that  of  the  auxiliary  process 
Zn-  From  (4.2)  we  can  write 

Fe  .  o' 

F  =  U*  vk  U. 

VK  loo 


Now  using  (4.1)  we  have 


k  fC  .  01  'D.  0 

F  .  =  U*(  l  v+n  k"n  )  U 

VK  n=0  0  Oj  0  0 


k  rc  °" 

=  l  (U*  v+n 
n=0  0  0 

w  —  . 


D 

U)  (U* 

i 


0  0 


Fvk  =nI0  Mv+A-n- 


Mn-U* 


II 

0  0 


U  and  N  =U*|  "  l). 


But  by  the  Lemma  we  have 


(4.4) 


$  =  U* 


¥  0 
0  0 


U  and  V  =  U* 


't''1  0  u. 
_0  0 


Thus  we  observe  that  M  and  N  are  exactly  the  n-th  Fourier  coefficients  of 

n  n 

M 

<t>  and  <t>  respectively. 

A 

Summarizing,  we  have  shown  that  the  best  linear  predictor  can  be 
written  exactly  in  the  same  form  obtained  in  [2]  for  the  full-rank  processes. 


i.e.  we  have 


X  =  1(1  M  Dk  )X_k,  In  Hq. 
u  k=0  n=0  v+n  *  "  * 


where  M  and  N  are  the  n-th  Fourier  coefficients  of  $  and  Its  generalized 
n  n 

inverse  4>#  (instead  of  $  and  its  inverse  <i>_1  in  the  full-rank  case). 
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